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1.  Introduction 

The  ability  to  detect  and  avoid  obstacles  is  a prerequisite  for  the  success  of  the  Demo  III 
Unmanned  Ground  Vehicles  (UGV)  program.  The  near  term  goals  of  the  project  are  to  be 
able  to  drive  the  High  Mobility  Multipurpose  Wheeled  Vehicle  (HMMWV) 
autonomously  on  cross-country  roads  with  the  ability  to: 

1.  Drive  autonomously  at  speeds  up  to  10  m.p.h.,  controlled  as  appropriate  for  the 
terrain  and  vehicle  dynamics. 

2.  Detect  obstacles  and  rough  terrain  conditions  in  time  to  enable  deceleration  to  a safe 
speed  or  the  ability  to  steer  around  the  problem  condition. 

3.  Maintain  a control  station  with  an  interface  for  limited  mission  planning  and  data 
collection. 

This  paper  discusses  an  obstacle  detection  algorithm  developed  at  NIST  in  support  of  the 
obstacle  detection  and  rough  terrain  conditions.  The  algorithm  is  a hybrid  of  grid-based 
and  sensor-based  obstacle  detection  and  mapping  techniques.  The  perception  and  obstacle 
detection/mapping  module  is  part  of  the  integrated  4D-Realtime  Control  System  (RCS) 
system  [1][2].  It  consists  of  two  sections:  an  obstacle  detection  section  and  a mapping 
section.  The  obstacle  detection  section  processes  range  data  read  from  a Ladar  sensor. 
The  algorithm  converts  range  data  into  Cartesian  coordinates  in  the  sensor  coordinate 
frame,  and  uses  this  information  to  detect  obstacles.  The  second  section,  the  mapping 
module,  projects  obstacle  points  onto  a grid-based  map.  The  map  is  used  by  the  4D-RCS 
planner  module  [11]  to  generate  a traversable  path  for  the  vehicle. We  have  demonstrated 
autonomous  driving  with  obstacle  detection  and  avoidance  on  the  NIST  grounds  and  the 
Nike  site  at  speeds  of  up  to  24  km/h. 
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Section  2 describes  the  sensors  used  in  the  4D-RCS  autonomous  driving  system.  Section 
3 describes  in  more  detail  the  Ladar  sensor’s  characteristics.  Section  4 describes  the 
obstacle  detection  algorithm.  Section  5 evaluates  the  algorithm  performance  on  both 
artificial  and  natural  obstacles.  Section  6 briefly  describes  the  obstacle  mapping  module. 
In  Section  7,  we  present  our  conclusions  and  plans  for  future  work. 

2.  System  Sensors 

The  sensors  used  in  the  system  include  a Domier  1.2  Hz.  Ladar  Range  Imaging  Camera 
(EBK);  the  Ashtech  Z12,  a Global  Positioning  System  (GPS)  sensor,  and  an  Inertial 
Navigation  System  (INS)  sensor  [14].  The  inertial  sensor  is  the  U.S.  Army’s  Modular 
Azimuth  and  Positioning  System  (MAPS)  which  contains  three  ring  laser  gyros,  three 
accelerometers,  and  a rear  axle  odometer.  All  sensors  are  mounted  on  the  HMMWV 
shown  in  Figure  1.  The  vehicle  is  equipped  with  electric  actuators  added  to  the  steering, 
brake,  transmission,  transfer  case,  and  parking  brake.  Dashboard-type  feedback  provides 
the  controller  with  RPM,  speed,  temperature,  fuel  level,  etc.  Multiple  navigation  sensors 
are  used.  A Kalman  filter  [9]  computes  vehicle  position  and  orientation  using  data  from 

an  inertial  dead  reckoning  system  and  a carrier  phase  differential  GPS  unit,1 


1 . Certain  commercial  equipment,  instruments,  or  materials  are  identified  in  this  paper  in  order  to 
adequately  specify  the  experimental  procedure.  Such  identification  does  not  imply  recommen- 
dation or  endorsement  by  NIST,  nor  does  it  imply  that  the  materials  or  equipment  identified  are 
necessarily  best  for  the  purpose. 
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Figure  1.  NIST  HMMWV 


3.  The  EBK  Ladar  sensor 

We  are  using  the  EBK  Ladar  sensor  as  the  primary  obstacle  detection  sensor  for 
autonomous  driving.  Table  1 shows  a summary  of  the  characteristic  parameters  of 
Dormer’s  1.2  Hz  Ladar  camera.  The  vertical  pixel  spacing  is  mechanically  fixed  at  0.5 
degrees  by  the  spacing  of  the  fiber  optic  array.  This  gives  a vertical  field  of  view  of  31.5 
degrees.  The  exact  horizontal  field  of  view  and  pixel  spacing  may  vary  slightly  because 
the  horizontal  and  vertical  scans  are  not  mechanically  linked.  However,  they  can  be 
calculated  exactly  from  scan  mirror  position  data  encoded  in  the  image.  The  nominal 
value  for  the  horizontal  field  of  view  is  60  degrees.  In  addition,  there  may  be  localized 
variations  of  the  horizontal  angular  position  of  the  image  pixels  due  to  mechanical 
disturbances  of  the  sensor,  but  these  can  be  detected  and  accounted  for  in  range 
computations.  The  range  image  is  generated  by  scanning  a laser  beam  over  the  field  of 
view.  At  each  point  in  the  scan,  a pulse  of  laser  light  is  emitted.  This  pulse  is  reflected 
back  to  the  sensor  from  the  object  in  its  line  of  flight.  The  range  of  the  object  can  be 
calculated  from  the  light  pulse’s  time  of  flight.  Because  the  measurement  uses  time  of 
flight  rather  than  phase  detection,  the  sensor  produces  an  absolute  value  with  no  range 
ambiguity.  Due  to  the  short  duration  of  the  laser  pulse,  the  EBK  is  eye-safe. 


The  laser  is  scanned  horizontally  across  the  field  of  view  by  an  oscillating  mirror.  The 
beam  is  scanned  vertically  by  a nutating  mirror  which  reflects  the  beam  through  a circular 
fiber  optic  array.  This  fiber  optic  array  converts  the  circular  motion  of  the  nutating  mirror 
into  a linear  scan  in  the  image  plane.  Additional  technical  information  about  the  EBK  can 
be  found  in  [3]  [4]. 

3.1  Effects  of  Different  Materials  on  Range  Data 

We  have  examined  the  sensor’s  ability  to  measure  different  materials  in  order  to  determine 
its  sensitivity.  In  order  for  the  sensor  to  receive  a laser  pulse  reflection,  the  reflection  must 
be  diffuse,  i.e.,  the  reflected  laser  light  must  travel  in  all  directions  from  the  target,  one  of 
which  is  back  to  the  sensor.  A highly  specular  surface  may  produce  a false  range  reading. 

3.1.1  Absorptive  Materials 

A false  detection  can  occur  when  the  surface  of  an  object  absorbs  so  much  of  the  incident 
light  that  the  returned  light  pulse  does  not  have  enough  energy  to  trigger  the  receiver. 
Examples  of  absorptive  materials  are  black  clothing,  dark  hair,  painted  black  stripes,  the 
black  baseboard  along  a wall,  and  very  dark  tar  stains  on  asphalt.  The  sensor  does  not 
receive  a reflected  signal  from  these  objects  within  the  time  of  flight  window,  which  is  set 
for  a 50  m maximum  range.  When  no  reflected  light  pulse  is  detected,  the  invalid  timing 
results  in  an  invalid  range  value.  Some  black  colored  objects,  however,  do  show  up. 
Automobile  tires  and  black  pavement  return  sufficient  light  to  trigger  the  detector.  An 
object’s  visibility  depends  on  its  reflective  properties  at  910  nm.  In  practice,  most 
surfaces  in  natural  and  artificial  scenes  offer  a return  signal  of  sufficient  intensity  to 
calculate  a range  value. 
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Table  1 EBK  operational  parameters 


EBK  Characteristic  Parameters 

Laser  Wavelength 

910  nm 

Optical  peak  power 

10W 

Imaging  Range 

5m  to  50  m (typical) 

Range  Resolution 

0.06  m 

Frame  Rate 

1 Hz 

Field  of  View 

3 1 degrees  by  60  degrees 

Beam  Divergence 

0.2  degrees 

Beam  Spacing 

0.5  degrees 

Eye  safe 

class  1,  IEC  825  and  DIN  VDE  0837 

Scanning 

Vertical:  fiber  optics 

Horizontal:  oscillating  mirror 

Operating  Voltage 

24  VDC  (18  V - 28  V) 

Power  Consumption 

110  W 

Data  Interface 

RS422  serial,  10  MBit/sec 

Weight 

15  kg 

Dimensions 

30  cm  x 28  cm  x 27  cm 

Operating  Environment 

IP65,  -10°  C to  50°  C 

3.1.2  Specular  Materials 

Highly  reflective  objects  give  rise  to  specular  reflections.  A good  example  of  this  is  a lake, 
which  the  sensor  cannot  detect.  Wet  pavement  also  does  not  register  well  with  the  sensor. 
When  the  Ladar  scans  a shiny  cylindrical  metallic  object,  only  the  surface  of  the  cylinder 
whose  normal  vector  is  parallel  with  the  sensor’s  scanning  vector  produces  a good  range 
reading.  All  other  laser  pulses  are  reflected  away  from  the  scanner.  However,  these 
reflections  may  bounce  off  of  other  objects  and  eventually  return  to  the  sensor. 
Consequently,  ranges  to  reflective  surfaces  may  appear  greater  than  their  true  value. 
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In  nature,  the  primary  source  of  specular  reflections  is  water.  Free  standing  water,  such  as 
lakes  and  puddles,  almost  completely  reflect  the  laser  beam  from  the  sensor.  In  these 
cases,  no  return  from  the  water’s  surface  is  detected.  This  appears  as  a “no  return  value” 
in  a range  images  The  cause  of  this  failure  is  either  the  sensor  seeing  a reflection  of  the 
sky  (out  of  range)  or  the  water  absorbing  the  incident  beam. 

The  signal  returned  from  wet  surfaces  varies  with  distance  and  incident  angle.  Wet 
pavement  tends  to  disappear  at  close  range  and  far  away  where  the  incident  angle  of  the 
sensor  beam  is  small.  However,  a good  signal  is  returned  in  the  mid  range  of  25  m. 

The  EBK  can  measure  range  readings  from  ice  and  snow.  However,  Dornier  has  reported 
that  partially  thawed  and  refrozen  snow  produces  a less  favorable  result. 

3.1.3  Vegetation 

Vegetation  poses  the  greatest  problem  to  obstacle  detection  using  range  data.  This  is 
because  range  data  gives  very  little  information  about  the  solidity  of  an  object  or  its 
traversability.  This  is  especially  a problem  for  off  road  driving.  In  addition,  vegetation 
can  obscure  other  vehicle  hazards,  such  as  rocks  or  ditches  hidden  in  tall  grass,  making  the 
hazards  more  difficult  to  detect. 

3.1.4  Fog 

The  910  nm  wave  length  of  the  laser  used  by  the  EBK  falls  in  the  near  infrared  portion  of 
the  electromagnetic  spectrum.  It  was  chosen  by  Dornier  for  its  good  reflective  properties. 
Being  close  to  the  visible  spectrum,  the  laser  is  affected  by  fog.  The  back  scattering  effect 
caused  by  heavy  fog  causes  a range  value  of  approximately  5 m to  be  returned  by  the 
sensor. 
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4.  Obstacle  Detection  Module 

The  purpose  of  the  obstacle  detection  module  is  to  extract  areas  which  cannot  or  should  not  be 
traversed  by  the  HMMWV.  Rocks,  fences,  trees,  and  steep  slopes  are  examples  of  obstacle  areas. 
Many  approaches  for  extracting  obstacles  from  range  images  have  been  proposed  The  most 
common  approach  is  to  fit  a plane  surface  to  a patch  of  points  [6]  [8]  [16]  in  a local  grid 
representation.  For  example,  Hebert  [6]  and  Kelly  [10]  have  used  the  approach  of  fitting  planar 
surfaces  to  a patch  of  points  to  detect  obstacles.  Their  algorithms  are  simple  and  fast  but  are  not 
very  robust:  small  sensor  position  errors  result  in  false  indications  of  obstacles.  We  propose 
using  differential  depth  and  slope  in  sensor  space  in  an  improved,  fast  obstacle  detector.  The 
algorithm  was  inspired  by  Veatch  and  Davis  [15]  and  Lux  and  Schaefer[12].  The  details  of  this 
algorithm  and  an  analysis  of  its  robustness  using  actual  outdoor  data  are  given  in  the  following 
subsections. 

4.1  The  Obstacle  Detection  Algorithm 

The  data  acquisition  time  for  a complete  Ladar  image  is  0.82  seconds.  The  image  consists  of  128 
vertical  scan  lines,  each  containing  64  range  values.  The  sensor  geometry  was  described  in 
Section  3.  Approximately  half  of  the  acquisition  time  is  needed  to  read  8192  range  values.  The 
remaining  time  is  needed  for  the  back-scan.  We  process  range  data  as  they  are  read,  rather  than 
waiting  for  a full  image.  In  this  way,  we  can  detect  an  obstacle  as  soon  as  the  data  are  available. 

As  each  scan  line  is  read,  a filter  is  applied  to  the  data.  The  filter  removes  outlier  points  caused  by 
specular  reflections,  e.g.  water,  or  reflections  within  the  sensor.  After  the  data  are  filtered,  the 
Cartesian  (x,  y,  z)  location  of  each  pixel  in  the  scan  line  is  computed  in  a coordinate  system 
centered  in  the  EBK  sensor.  Each  pixel  contains  x,  y,  z and  range(r)  values  which  are  used  to 
compute  depth  derivatives  and  surface  slopes.  The  criteria  used  for  detecting  obstacles  are: 
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1 . A discontinuity  in  elevation  exceeding  some  value  T 

2.  The  surface  slope  exceeding  some  value,  or. 

These  conditions  are  expressed  mathematically  in  the  following  way: 

th 

Let  p(i,  j),  be  the  i position  pixel  in  the  j scan  line  of  the  EBK  image. 

Let  x(i),  y(i),  z(i)  be  the  Cartesian  coordinates  of  this  pixel. 

Let  r(i)  be  the  range  value  at  this  pixel. 

For  every  p((k,  j),  x(k),  y(k),  z(k),  and  r(k))  where  (i-n)  <=  k <=  (i+n) 

(n  = 2 in  our  experiments) 
p((k,  j),  x(k),  y(k),  z(k),  r(k))  is  voted  an  obstacle  if: 

\z(i)  - z(k)\  > T and 

-z{k))2/[{x(i)  -x{k))~  + (y(i)-y(k))2  + (z(i)  - z(k))2])  > a" 

To  avoid  frequent  false  positive  detections,  a local  neighborhood  of  each  pixel  votes  on  whether  or 
not  a pixel  is  an  obstacle.  In  practice,  a neighborhood  of  five  pixels  is  used.  Each  pixel  in  the  scan 
line  can  receive  up  to  2n  votes  where  n is  the  neighborhood  size.  The  higher  a pixel’s  vote,  the 
higher  the  confidence  that  the  pixel  is  an  obstacle.  If  the  confidence  is  greater  than  some  threshold 
value,  the  pixel  is  labelled  as  an  obstacle.  We  empirically  find  a confidence  threshold  value  of  2 
for  our  experiments. 

5.  Analysis  and  Evaluation  of  the  Obstacle  Detection  Algorithm 

Two  types  of  tests  were  performed  to  analyze  and  evaluate  the  obstacle  detection  algorithm.  Both 
scenarios  involved  collecting  sets  of  Ladar  data  which  were  analyzed  at  a later  time.  The  first  set 
of  tests  was  performed  at  the  NIST  Nike  Test  Site  in  Gaithersburg,  Maryland.  The  obstacles 
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tested  were  artificial,  i.e.,  known  size  blocks  of  wood  arranged  in  different  groupings.  The 
second  set  of  tests  was  performed  at  the  U.  S.  Army  Aberdeen  Proving  Grounds  (APG)  in 
Aberdeen,  Maryland.  This  set  of  data  was  collected  at  a surveyed  site  that  included  rocks 
and  ditches. 

5.1  Artificial  Obstacles 

Our  objective  at  the  Nike  site  was  to  evaluate  the  algorithm’s  ability  to  detect  obstacles  of 
known  sizes  and  to  determine  the  maximum  range  at  which  different  size  obstacles  could 
be  detected.  The  obstacles  consisted  of  sections  of  61  cm  x 15  cm  lumber  pre-cut  into  10 
cm,  15  cm  and  20  cm  sections.  These  blocks  were  grouped  together  to  form  obstacle 
groups  of  varying,  but  known,  dimensions.  The  obstacles  were  placed  on  a road  (Figure 
2),  and  Ladar  data  were  collected  as  the  HMMWV  approached  them.  Figures  3.  4 and  5 


are  sets  of  processed  Ladar  images  showing  the  results  of  the  obstacle  detection 
algorithm  on  groups  of  different  size  blocks.  The  blocks  in  Figure  3 are  20  cm  wide  by  15 
cm  high.  The  blocks  in  Fig.  4 are  30  cm  wide  by  20  cm  high.  In  Fig.  5,  the  blocks  are 
25  cm  wide  by  15  cm  high.  The  left  image  in  each  set  is  the  farthest  range  at  which  the 
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obstacles  were  detected;  the  right  image  is  the  same  set  of  obstacles  at  a closer  range.  In  Fig.  3, 
only  one  block  is  detected  at  17.5  m while  all  blocks  are  detected  at  9.5  m.  In  Fig.  4,  2 blocks  are 
detected  at  22  m,  and  4 blocks  are  detected  at  13.2  m.  In  Figure  5,  3 blocks  are  detected  at  19  m, 
and  4 blocks  are  detected  at  15.7  m. 

Because  of  the  sensor  design  (see  Section  3),  the  probability  of  the  laser  beam  hitting  a target  is  a 
function  of  both  the  size  of  the  target  and  its  range  from  the  sensor.  This  relationship  is  shown  in 
the  graphs  plotted  in  Figure  6.  In  each  of  the  plots,  the  horizontal  axis  measures  range  in  meters 
from  the  sensor.  The  vertical  axis  is  the  probability  that  the  Ladar  beam  will  hit  the  target.  In 
Figure  6a,  the  probability  of  the  beam  hitting  the  target  increases  to  100  % at  a distance  of  27.5  m 
and  remains  at  this  level  through  the  remaining  valid  data  range.  In  Figure  6b,  the  probability 
increases  to  100  % at  36  m and  remains  at  100  % for  ranges  less  than  36  m.  The  same  is  true  in 
Figure  6c  for  ranges  between  5 m and  47.8  m.  The  target  sizes  in  Fig.  6a,  6b, and  6c  are  15  cm  by 
15  cm,  20  cm  x 20  cm,  and  25  cm  x 25  cm  respectively.  In  addition  to  the  blocks  of  wood,  grass 
and  trees  along  the  side  of  the  road  are  also  detected  as  obstacle  pixels. 
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Obstacle  detection  at  17.5  m Obstacle  detection  at  9.5  m 

Figure  3 Target  Size:  1 5 cm  x 20  cm(width  x height) 


Obstacle  detection  at  22  m 

Figure  4 Target  Size 


Obstacle  detection  at  19  m 

Figure  5 Target  Size: 
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Obstacle  detection  at  13.2  m 
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Obstacle  detection  at  15.7  m 
25  cm  x 15  cm 
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PROBABILITY  OF  BEAM  HITTING  TARGET 


Figure  6:  Probability  plots  of  ladar  beam  hitting  target. 

(a)  Target  size  15  cm  x 15  cm  (b)  Target  size  20  cm  x 20  cm  (c)  Target  size  25  cm  x 25  cm 


5.2  Natural  Obstacles 

In  this  section,  we  describe  our  analysis  of  natural  obstacles.  Our  objective  was  to  determine  the 
farthest  range  at  which  pixels  representing  depressions  could  be  detected.  The  data  collection 
site  was  the  Perryman  site,  located  in  the  U.  S.  Army  Aberdeen  Proving  Grounds  in  Aberdeen, 
Maryland.  In  addition  to  the  Ladar  data,  sets  of  images  from  3 pairs  of  stereo  cameras  were 

also  collected.  The  stereo  data  were  analyzed  by  scientists  at  NASA’s  Jet  Propulsion 
Laboratory  (JPL)  using  algorithms  designed  for  stereo  inputs  [13].  This  report  discusses  only 

2.  A complete  description  of  the  data  collection  can  be  found  at: 

http://isd.cme.nist.gov/staff/coombs/proj/mobility/apg-data/doc/datanotes.html. 
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the  results  of  the  Ladar  obstacle  detection  algorithm. 


Figure  7 is  a map  of  the  sections  of  Perryman  in  which  negative  obstacle  (ditch)  data  were 
collected.  The  ditches  were  dug  by  a back  hoe.  Ditch  1 is  0.61  m wide;  Ditch  2 is  1.22  m 
wide.  Ditch  3 is  1.83  m wide,  and  Ditch  4 is  2.44  m wide.  The  lengths  of  all  the  ditches  are 
the  same.  This  data  was  used  to  evaluate  the  effectiveness  of  the  algorithm  in  detecting 

Ditch  Survey 

4,368,505 


Figure  7 Ditch  Survey  Site 


negative  obstacles  (depressions  in  the  ground  plane).  Figure  8 is  a black  and  white  image, 
digitized  from  video  tape,  showing  the  site  referred  to  in  the  ditch  survey  map.  The 
viewing  direction  in  Figure  8 is  along  the  direction  of  the  approach  vector  in  Figure  7. 
Data  were  collected  at  different  times  of  day  under  varying  weather  conditions.  A 


1.22  m l.83m  2.44  m 


■ - . - 


HI  I mmm 


i&m-- 


Figure  8Ditches  at  Perryman 


decrease  in  daylight  did  not  affect  the  Ladar  data  which  performed  at  the  same  level 
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during  dawn,  daylight,  and  evening  hours.  For  each  of  the  ditches,  the  HMMWV  was 
driven  at  speeds  of  8 kilometers  per  hour  (km/h),  16  km/h,  and  24  km/h  The  vehicle’s 
approach  to  the  ditches  also  varied:  some  runs  were  perpendicular  to  the  ditches;  some 
were  oblique.  The  output  of  the  obstacle  detection  algorithm  was  qualitatively  evaluated. 
Grouping  of  obstacle  pixels  into  objects  is  planned  at  a future  time  and  was  not  a factor  in 
this  evaluation.  Figure  9 is  a set  of  images  showing  the  unprocessed  Ladar  data  (Figure 
9a)  and  the  processed  data  (Figure  9b)  of  a sample  image  taken  from  the  “ditch  1"  data 
set.The  raw  data  is  encoded  as  grey  scale  values  such  that  darker  grey  values  represent 
ranges  farther  from  the  sensor.  White  pixels  in  the  processed  image  represent  pixels 
labelled  as  obstacle  points.  Although  it  is  very  difficult  to  recognize  the  ditch  in  the  raw 
image,  the  ditch  pixels  are  detected  in  the  processed  image  The  ditch  is  10  m from  the 
vehicle  in  Figure  9. 


(a)  (b) 


Figure  9 Ditch  1:  Raw  and  processed  ladar  data 

Figure  10  is  another  example  of  negative  obstacle  detection.  This  image  set  shows  raw 
and  processed  data  from  the  “ditch  4"  data  sequence.  Again,  the  pixels  associated  with  the 
ditches  are  extracted.  In  this  figure,  the  ditch  is  15  m from  the  vehicle. 

Table  2 shows  the  results  of  the  obstacle  detection  algorithm  on  the  ditch  data  as  the 
vehicle  approached  the  ditches  in  the  normal  direction.  The  distances,  measured  in 
meters,  represent  the  farthest  distance  at  which  the  algorithm  extracted  pixels  representing 
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Figure  10  Ditch  4:  Raw  and  processed  ladar  data 


ditch  data.  The  pixels  associated  with  the  ditch  are  qualitatively  recognizable  to  a trained 
eye. 

As  an  aid  to  understanding  these  values,  it  is  helpful  to  understand  that  the  minimum 
stopping  distance  required  for  a vehicle  travelling  at  5 m.p.h.  (assuming  a relatively  flat 
ground  surface)  is  approximately  10  meters.  At  10  m.p.h.,  the  stopping  distance  is  20 
meters,  and  at  15  m.p.h.,  it  is  30  meters. 


Table  2 NEGATIVE  OBSTACLE  DETECTION-  NORMAL 

APPROACH 


Ditch  1 

Ditch  2 

Ditch  3 

Ditch  4 

5 mph. 

10.0  m 

19.0  m 

17.0  m 

23.0  m 

10  mph. 

10.0  m 

14.0  m 

20.0  m 

20.7  m 

15  mph. 

9.5  m 

15.0  m 

20.5  m 

21.2  m 

Figure  1 1 is  a map  of  the  area  of  Perryman  containing  the  rock  obstacles.  The  rocks  were 
placed  both  on-road  and  off-road.  The  rocks  labelled  “rock  1”  and  “rock  2”  are 
approximately  15  cm  high  and  25  cm  wide.  Rocks  3 and  4 are  approximately  30  cm  high 
and  30  cm  wide.  Rock  5 is  approximately  45  cm  high  and  30  cm  wide.  The  rock  cluster  is 
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North  ( 


approximately  45  cm  high  and  100  cm  wide. 


East (m) 
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Figure  12  is  a video  image  showing  rocks  1,  2,  3 and  4. 


Figure  13  is  a magnified  video  image  showing  the  rock  cluster. 


Figure  14a  shows  a Ladar  scene  of  rocks  1 and  2 and  Figure  14b  the  obstacles  detected  in 
this  image.  Rocks  1 and  2 are  highlighted  by  white  boxes  in  the  right  image. 


(a) Raw  ladar  data  (b)  Rock  1 at  a distance  of  1 1 .5  m 

Rock  2 at  a distance  of  19.2  m 

Figure  14  (a)  Raw  ladar  image  of  rocks  1 and  2 
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Figure  15  shows  raw  (Fig.  15a)  and  processed  (Fig.  15b)  Ladar  image  data  for  rocks  3 and 


4 with  boxes  highlighting  the  rocks. 


(a)Raw  ladar  data  (b)  Rock  3 at  a distance  of  18.4  m 

Rock  4 at  a distance  of  25.0  m 

Figure  15  Rocks  3 and  4 


Figure  16a  is  a Ladar  scene  of  the  rock  cluster.  Figure  16b  shows  the  extracted  obstacle 
points.  The  direction  of  approach  for  Figs.  14,  15,  and  16  is  from  Northeast  to  Southwest 
in  Fig.  11.  Table  3 is  a summary  of  the  results  of  applying  the  obstacle  detection 


(a)  Raw  ladar  data  (b)  Rock  cluster  at  a distance  of  25  m 

Figure  16  Rock  Cluster 


algorithm  to  the  data  sets  collected  at  different  times  of  day.  The  variability  of  detection  is 
a result  of  the  following  factors: 

(1)  Data  acquisition  rate  (1  image  per  second). 

(2)  Relationship  between  vehicle  velocity  and  the  time  an  image  is  acquired.  If  the 
vehicle  is  moving  quickly,  the  scanning  Ladar  beam  may  miss  the  obstacle  completely. 

(3)  Difficulty  in  identifying  the  targeted  obstacle  when  it  is  hidden  in  tall  grass. 
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Table  3 Detection  of  Positive  Obstacles 


Rockl 

Rock2 

Rock3 

Rock4 

Rock5 

Rock 

Clump 

5 mph 

13.9  m 

20.2  m 

18.6  m 

25.0  m 

12.8  m 

31.8  m 

10  mph 

13.2  m 

20.5  m 

22.3  m 

24.1  m 

17.0  m 

26.0  m 

15  mph 

15.0  m 

23.2  m 

22.2  m 

26.6  m 

13.7  m 

30.6  m 

The  need  for  recognizing  and  classifying  obstacles  is  clear  from  the  outputs  shown  in 
these  figures.  In  addition  to  the  rocks  and/or  rock  cluster,  the  algorithm  has  flagged  tall 
grass,  bushes,  and  background  trees  as  obstacles.  Our  future  work  includes  plans  to 
fuse  Ladar  images  with  video  images  to  generate  windows  of  interest  in  the  video  images. 
Classification  of  obstacles  will  be  based  on  the  information  in  the  video  windows. 


6.  Obstacle  Map  and  List  Module 

We  have  adapted  Kelly’s  [10]  grid  obstacle  map  for  representing  obstacles  for  path 
planning  and  vehicle  control.  The  map  representation  contains  terrain  surface  slopes, 
uncertainty  measures,  average  depth  information,  and  a list  of  obstacles  for  every  grid  in 
the  map.  The  path  planning  algorithm  currently  implemented  in  the  4D-RCS  system  [11] 
uses  only  a subset  of  the  information  contained  in  the  map.  It  generates  traversability 
paths  from  the  map  by  using  a binary  flag  indicating  whether  or  not  the  grid  is  an  obstacle, 
a timestamp  indicating  when  an  obstacle  was  last  updated,  and  an  obstacle  confidence 
value. 


7.  Conclusion 

We  have  described  an  algorithm  designed  to  detect  obstacles  in  Ladar  imagery.  The 


20 


■ 


algorithm  is  one  of  a suite  of  algorithms  being  tested  in  order  to  evaluate  different  sensors 
that  might  be  used  for  obstacle  detection  in  the  UGV  program.  We  have  characterized  the 
Ladar  sensor  being  used,  and  described  its  performance  on  different  types  of  surfaces. 

In  order  to  evaluate  the  obstacle  detection  algorithm,  we  collected  extensive  sets  of  Ladar 
data.  These  data  were  used  as  input  for  the  obstacle  detection  algorithm.  The  algorithm 
was  able  to  detect  both  positive  (rocks)  and  negative  (ditches)  obstacles.  Detectability  was 
determined  to  be  a function  of  both  the  range  to  the  obstacle  and  the  size  of  the  obstacle. 
The  results  of  the  evaluation  tests  are  reported. 

The  obstacle  detection  algorithm  was  also  tested  in  a real-time  demonstration  as  part  of 
the  4D-RCS  control  system.  In  conjunction  with  the  planner  module  and  behavior 
generation  modules  [1],  we  were  able  to  detect  obstacles  on  both  the  NIST  grounds  and 
the  NIKE  test  site  while  driving. 
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